Observation scoring PR - #321
Conversation
…alues to have final dimension even if scalar
DanWaxman
left a comment
DanWaxman
left a comment
There was a problem hiding this comment.
Few small comments. I also opened #323 for some other changes.
In general, I find it a bit sad we can't use this with LatentPathBuilder. I guess this is fine as an intermediate draft so we get the feature out, but I'd like to have a more general Simulator-based inference interface here. This should enable use with LPB and scoring at lead time. But again happy to boot that to some other time in the near future.
| try: | ||
| if isinstance(rule, gaussian_rules): | ||
| if predictions.mean is None: | ||
| raise NotImplementedError( | ||
| _missing_prediction_error(rule.site_name, "mean", "") | ||
| ) | ||
| if predictions.obs_cov is None: | ||
| raise NotImplementedError( | ||
| _missing_prediction_error(rule.site_name, "obs_cov", "") | ||
| ) | ||
| score_arrays[rule.site_name] = rule.compute( | ||
| obs_values=obs_arr, | ||
| pred_mean=predictions.mean, | ||
| pred_cov=predictions.obs_cov, | ||
| ) | ||
| elif isinstance(rule, EnergyScore): | ||
| score_ensemble = _select_scoring_ensemble( | ||
| predictions, | ||
| scoring_config=scoring_config, | ||
| rule_name=rule.site_name, | ||
| ) | ||
| score_arrays[rule.site_name] = rule.compute( | ||
| obs_values=obs_arr, | ||
| pred_mean=predictions.mean, | ||
| pred_cov=predictions.obs_cov, | ||
| pred_ensemble=score_ensemble, | ||
| sample_seed=scoring_config.sample_seed, | ||
| ) | ||
| else: | ||
| raise NotImplementedError( | ||
| f"Unsupported observation scoring rule type: {type(rule).__name__}." | ||
| ) | ||
| except NotImplementedError: | ||
| if scoring_config.unsupported == "skip": | ||
| continue | ||
| raise |
There was a problem hiding this comment.
I don't fully get this API choice, i.e., to allow for unsupported scores to be requested.
There was a problem hiding this comment.
Not really a comment about anything in particular, but I'm surprised this is such a big file!
There was a problem hiding this comment.
Reduced from 265 -> 215
* Add deprecration notice for diagnostics/plotting_utils * Doc changes
… passing information directly when available
2 participants
Supersedes #296